Mobile devices or apparatus such as mobile phones are becoming equipped with greater number of sensors such as microphones or cameras that enable the user of the apparatus or device to perform a much greater range of operations. For example mobile apparatus or devices can be equipped with a cluster of microphones (otherwise known as an array of microphones) which can be used to perform source separation to extract individual audio sources from the sound field surrounding the apparatus. The operation of source separation produces an estimate of the independent or uncorrelated sources. Alternatively using the array of microphones beamforming, an audio signal processing technique, can be used to improve the signal to interference ratio of these directional sources.
These individual sources are often re-synthesised for playback on headphones or loudspeaker sets. Typically the direction of arrival for each of the sources is estimated. A binaural synthesis on each of the sources can be performed based on the directional of arrival (DOA). The binaural output of different sources can then be added together to create a binaural rendition or representation of the sound field. Similar synthesis can also be performed for multi-loudspeaker playback.
Mobile devices are furthermore often being equipped with a camera and in some cases more than one camera in order to generate three-dimensional image data that can be viewed. These array camera-based devices (such as apparatus comprising two cameras or a camera and a depth sensor) can be used to estimate a depth map for the field of view. The accuracy of the depth map can be coarse or fine depending on the type of sensors used and the resolution of the sensors. The depth map is often used to synthesise 3-D images or perform refocusing, however the usage of depth information is often limited to image rendering.